Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey

Large, pre-trained language models (PLMs) such as BERT and GPT have drastically changed the Natural Language Processing (NLP) field. For numerous NLP tasks, approaches leveraging PLMs achieved state-of-the-art performance. The key idea is to learn a generic, latent representation of from generic task once, then share it across disparate tasks. modeling serves task, one with abundant self-supervised text available for extensive training. This article presents fundamental concepts PLM architectures comprehensive view shift PLM-driven techniques. It surveys work applying pre-training fine-tuning, prompting, generation approaches. In addition, discusses limitations suggested directions future research.